Method for converting input image data into output image data, image conversion unit for converting input image data into output image data, image processing apparatus, display device

ABSTRACT

In a method, unit and display device, the input image signal is split into a regional contrast signal and a detail signal, followed by stretching separately the dynamic ranges for at least one of the signals. The dynamic range for the regional contrast signal is stretched with a higher stretch ratio than the dynamic range for the detail signal. The stretch ratio for the detail signal may be near 1 or 1. Further, highlights are identified, and for the highlights the dynamic range is stretched to an even higher degree than for the regional contrast signal

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of prior U.S. patent application Ser.No. 13/254,191, filed Sep. 1, 2011, which is a national application, ofPCT Application No. PCT/IB2010/050905, filed Mar. 3, 2010 and claims thebenefit of European Patent Application No. 09154549.1, filed Mar. 6,2009, the entire contents of each of which are incorporated herein byreference thereto.

FIELD OF THE INVENTION

The invention relates to a method for converting input image data intooutput image data.

The invention further relates to an image conversion unit for convertinginput image data into output image data.

The invention further relates to an image processing apparatuscomprising:

receiving means for receiving input image data,

an image conversion, unit for converting input image data into outputimage data.

The invention further relates to a display device comprising an imageprocessing apparatus comprising:

receiving means for receiving input image data,

an image conversion unit for convening input image data into outputimage data.

BACKGROUND OF THE INVENTION

To enable an acceptable representation of high-dynamic range (HDR)imagery on a display with a dynamic range that is typically severalorders of magnitude lower, the dynamic range of recorded video sequencesis usually compressed by means of tone-mapping during acquisition andtransmission. The dynamic range of many outdoor scenes can be as largeas 12 orders of magnitude, whereas most liquid crystal displays (LCDs)merely offer a static contrast ratio of about 3 orders of magnitude. Asa result severe dynamic range compression is required in the earlystages of the imaging pipeline to enable a pleasant representation ofthe scene on a LDR (low dynamic range) display. Using simple techniquesusually has the drawback that the contrast of small details can becompromised or even lost.

To address these shortcomings, more advanced adaptive methods have beendeveloped. These methods predominantly compress large-scale contrastswhile preserving the contrast of fine details.

This approach performs well as long as the display system's capabilitiesremain more or less similar to those anticipated during compression inthe early stages of the imaging pipeline. However, with newhigh-dynamic-range display systems, static contrast ratios of up to 6orders of magnitude can be achieved. Moreover, such display systems maybe capable of locally (in time or space) producing a very high peakbrightness. For example, this can be achieved by 2D dimmable LEDbacklights, where the power saved by dimming some LEDs underneath darkimage portions may be used to boost other LEDs underneath brightregions. An extension of the input LDR image data into a HDR imagesignal has been found to often result in an unnatural appearance of thescene.

SUMMARY OF THE INVENTION

It is an object of the invention to provide a method, conversion unitand image processing apparatus with an aim of increasing the quality ofreproduction and providing a more pleasant and natural appearance ofimages.

To this end the method in accordance with the invention is characterizedin that

The input image data is converted into at least two signals, a firstsignal providing regional contrast data and a second signal providingdetail data,

The dynamic range of at least the first signal is stretched, whereindynamic range of the first signal is stretched to a higher degree thanthe dynamic range of the second signal,

The stretched first and second signals are combined is an output signal.

The inventor has realised that the problems arise out of an imbalancebetween local and regional contrast. Preservation of detail contrastduring dynamic range compression during acquisition in combination withan overall dynamic range extension during or prior to display results inan enhancement of fine details relative to regional contrasts in thedisplayed image. The regional contrast data comprises relatively lowspatial frequency information. The detail data comprises higher spatialfrequency information.

For considerable extension factors, this results in an unnaturalappearance of the scene and could also lead to an undesiredamplification of analog and digital noise.

A possible solution would be to use, during range extension, themathematical inverse of the mapping operator used during rangecompression to retrieve the original HDR scene. This, however, wouldrequire knowledge of the used compression method, which would have to beincluded in the input signal. However, in practice, we often have todeal with legacy LDR video without knowledge of how its dynamic rangewas compressed during acquisition and encoding. This ‘perfect’ solutionis thus often not practical. Apart from this aspect, the receiving unitwould have to be able to match various possible compression methods.

The present invention provides a more balanced LDR to HDR conversion ofthe input image data into an output signal.

The input signal is split into a first signal providing regional,semiglobal data and a second signal providing the details. The firstsignal can for instance be made by low pass filtering the input signal,including low pass filtering methods which preserve edge features, suchas for instance bilateral filtering. The second signal providing detailscan be made by e.g. subtracting the first signal from the input datasignal.

At least the first signal is stretched, i.e. the dynamic range of atleast the first signal is extended. The two signals are differentlystretched, wherein the second signal is stretched to a smaller degreethan the first signal. This reduces the unnatural visible enhancement offine details relative to regional contrasts, resulting in a more naturalappearance of the scene. To some extent noise is also subdued. Inpreferred embodiments the second signal is not stretched. If during theoriginal compression the details were preserved, the second signalproviding detail information need not be stretched. This is a relativelysimple embodiment allowing a simplification of the algorithm.

In preferred embodiments the dynamic range of the combined stretchedfirst and second signal is bound by an upper value. This upper value maybe lower than the maximum allowable signal on the display. The inputimage signal is further analyzed to identify groups of pixels forminghighlights in the image and wherein the pixel data for said identifiedgroups of pixels are converted into a third signal such that the thirdsignal covers a dynamic range extending to above the said upper value toa upper maximum pixel value and wherein the third signal is combinedwith the combined stretched first and second signal.

The signal comprising the stretched first and second signal has adynamic range which is bound by an upper value. In the preferredembodiment above said upper value and to a maximum value, an upperdynamic range of pixel values is reserved for displaying highlights.

It has been found that, especially for very high luminance displays, themaximum achievable intensity is so high that the viewer, in a sense,becomes blinded by the light. In moderate cases, the viewer will onlyperceive the bright spots and will not, or only to a very limitedextent, be able to perceive the darker details of the scene. In extremecases, however, this can be painful or even harmful for the viewers'eyes. By limiting the range to which the combined first and secondsignal is stretched, this is avoided. However, this does not make fulluse of the possibilities of HDR displays. In preferred embodiments themaximum luminance is kept below the possibilities of a high luminancedevice. By identifying highlights in the image and placing their pixelvalues in the highest part of the dynamic range of the display, thesehighlights are brought to the forefront without blinding the viewerthereby providing a very crisp and clear image. In an embodiment thehighlights are identified by selecting groups of pixels with pixel valuein a range close to or at the upper value of the LDR range, wherein in aneighborhood of a high pixel value pixel the number of high pixel valuepixels is below a threshold, i.e. for small groups of high intensitypixels.

Highlights are relatively small groups of high intensity pixels. Thedynamic range of the display device above the upper value is populatedby the highlights. This has shown to provide a high quality imagewherein, on the one hand, the details are not unnaturally enhanced, orbright blinding spots appear in the image, while, on the other hand, thehighlights imaged at the high end of the display range provide for asparkling and crisp image.

In preferred embodiments the upper value of the dynamic range for thecombined stretched first and second signal lies in a range correspondingto light intensities when displayed on a display of 500 to 1000 Nit, andthe upper maximum pixel value lies in a range corresponding to lightintensities when displayed on a display of above 1000 Nit, preferablyabove 2500 Nit.

These and further aspects of the invention will be explained in greaterdetail by way of example and with reference to the accompanyingdrawings, in which

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 provides a schematic flow chart for an embodiment of theinvention;

FIG. 2 illustrates extension of dynamic ranges;

FIG. 3 illustrates a highlight identification algorithm;

FIGS. 4 a to 4 f illustrate the effects of a dynamic range extensionalgorithm according to the invention;

FIG. 5 illustrates a mixing map;

FIGS. 6 a to 6 c further illustrate dynamic range extension according tothe invention;

FIG. 7 a to 7 d and 8 a to 8 d provide further examples of dynamic rangeextension according to the invention;

FIG. 9 illustrates a display device according to the present invention.

The Figures are not drawn to scale. Generally, identical components aredenoted by the same reference numerals in the figures.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

It is remarked that below examples are shown.

The dynamic range of many outdoor scenes can be as large as 12 orders ofmagnitude, whereas most liquid crystal displays (LCDs) merely offer astatic contrast ratio of about 3 orders of magnitude. As a result,severe dynamic range compression is required in the early stages of theimaging pipeline to enable a pleasant representation of the scene on aLDR (low dynamic Range) display. The most straightforward approach todynamic range compression is by means of global tone-mapping operators.However, the main drawback of these simple techniques is that thecontrast of small details can be compromised. To address theseshortcomings, more advanced methods have been developed compressregional (large-scale) contrasts while preserving the contrast of finedetails.

On conventional LDR (low-dynamic range) display screens, the contrast ofthe imagery is usually stretched to the full capabilities of the displaydevice (i.e. 0 to black, 255 to white for an 8-bit system), subject touser preference, sometimes supported by a histogram stretch prior todisplay. This approach performs well as long as the display system'scapabilities remain more or less similar to those anticipated duringcompression in the early stages of the imaging pipeline. However, in newHDR (high-dynamie-range) display systems static contrast ratios of up to6 orders of magnitude are achieved. Moreover, such display systems maybe capable of locally (in time or space) producing a very high peakbrightness. For example, this can be achieved by 2D dimmable LEDbacklights, where the power saved by dimming some LEDs underneath darkimage portions may be used to boost other LEDs underneath brightregions.

When displaying legacy LDR video directly on a HDR display, an artifactoccurs, namely imbalance between local and regional contrast.

Preservation of detail contrast during range compression in combinationwith a range extension prior to display, results in an enhancement offine details relative to regional contrasts. For large extensionfactors, this results in an unnatural appearance of the scene andsometimes an undesired amplification of noise.

In the method in according with the invention the input image data isconverted into at least two signals, a first signal providing lowspatial frequency regional contrast data and a second signal providinghigh spatial frequency detail data. The dynamic range of at least thefirst signal is stretched, wherein the dynamic range of the first signalis stretched to a higher degree than the dynamic range of the secondsignal. The stretched first and second signals are combined in the imageoutput signal.

The first signal provides a regional contrast signal and the secondsignal provides a detail layer. The two signals are separatelystretched, wherein the first signal is stretched more than the secondsignal. In effect a regional stretch of the regional contrast signal isperformed, for instance by low pass filtering. Upon this stretching thelocal detail is stretched but to a lower degree. The two signals arecombined. This reduces, compared to an overall stretch of the incomingsignal, the imbalance between detail and regional image is reduced. Inpreferred embodiments the second signal is made by subtracting the firstsignal from the input image data.

FIG. 1 illustrates a flow diagram for an exemplary algorithm inaccordance with the invention.

The algorithm performs dynamic range extension as a dual signalprocedure. Initially, regional contrasts are extracted from the inputsignal V_(in) by applying, in this example, a low-pass filter 1 to thevideo, providing a first signal regional contrast signal V_(RC), andextracting a detail layer from the input signal V_(in), providing asecond detail signal V_(D). In this example V_(D) extracted by computingthe difference between the regional contrasts and the input insubtracter 2:

In formula:

V _(RC) =F _(bit)(V _(in)),

V _(D) =V _(in) −V _(RC),

where V_(in) denotes the input video and F_(bit) denotes the applicationof low pass filter, preferably a fast bilateral filter. Preferablybilateral filtering using a bilateral grid as the low-pass operator isexecuted. This approach provides a computationally efficientapproximation to the full bilateral filter. The main benefit of thismethod is that it provides a cheap edge-preserving blur filter, thuspreventing halo artifacts often associated with linear spatial filterkernels. Bilateral filtering using the bilateral grid can effectively besummarized as (1) constructing local histograms, (2) applying amultidimensional linear filter kernel to these histograms and (3)slicing (=interpolating) the desired output pixels. Although preferred,it should be noted that the bilateral grid does not represent anessential part of the current invention. Regional contrasts canalternatively be extracted using conventional (banks of) low-passfilters. Instead of using a mathematical algorithm to generate the firstand second signal other methods can also be used such as for instancepredefined special classes e.g. dark room interior.

To reduce the imbalance between regional and detail in an image of highluminance and maintain a natural balance between the fine detail andregional contrast when applying dynamic range extension, the two signalsV_(RC) and V_(D) are mapped separately. One preferred way of doing so isby stretching the regional contrast V_(HC) linearly from the inputdynamic range [K_(LDR)-W_(LDR)] to a pre-defined target dynamic range[K₀-W₀] which could depend on the display capabilities, the human eyecapabilities or personal preference:

${\overset{\sim}{V}}_{RC} = {{\left( {V_{RC} - K_{LDR}} \right) \cdot \frac{\left( {W_{0} - K_{0}} \right)}{\left( {W_{LDR} - K_{LDR}} \right)}} + {K_{0}.}}$

Wherein {tilde over (V)}_(RC) is the stretched signal. Such predefinedtarget dynamic range can be set by the manufacturer. W₀ defines theupper value of the dynamic range for the combined signal. In FIG. 1 thestretching of the first signal V_(RC) is schematically illustrated byM(V_(RC)) where M stands for a stretching operation of which the aboveformula is an example in which linear stretching is used. Non-linearstretching using other formulas for mapping are also possible, for thisstretching or mapping step, as well as for any other stretching ormapping step.

The stretching operation provides a range extension. Grosso modo foestretching of the dynamic range is a factor

$\frac{\left( {W_{0} - K_{0}} \right)}{\left( {W_{LDR} - K_{LDR}} \right)}$

being the ratio for the input dynamic range (W_(LDR)-K_(LDR)* and thetarget dynamic range, i.e. the amount of stretching applied to the firstregional contrast signal. The stretching is performed in stretcher 3.The stretcher 3 maps the incoming data V_(in) with an incoming dynamicrange (W_(LDK)-K_(LDR)) onto a stretched dynamic range (W₀-K₀).

In the above preferably W₀<W_(HDR) where W_(HDR) is the maximum range ofthe display, thereby keeping the predefined target dynamic range belowthe maximum display dynamic range. This prevents that large bright areaswill be imaged/rendered at unpleasantly high luminances.

Preferably W₀ is in the range corresponding to a luminance in the rangeof 500 to 1000 Nit.

Second, the detail layer signal is enhanced by applying a moderate,compared to the stretching factor of the first signal enhancement factorg_(D)in enhancer 4:

{tilde over (V)}_(D)=g_(D)V_(D)

{tilde over (V)}_(D) is the stretched second signal comprising thedetails. Preferably the gain g_(D) is close to 1, for instance in therange between 1 and 1.2, or simply 1, in the latter case the detaillayer data V_(D) is left as it is, without enhancement, which is asimple preferred embodiment. In many legacy compressed LDR signal, thecompression is performed which more or less maintain contrast indetails. Thus, leaving the detail layer unaffected i.e. applying a gainfactor of 1 is often sufficient and reduces the complexity of thealgorithm.

Obviously, extension functions M(V_(RC)) and g_(D) other than the abovesimple linear scaling can be used, such as power functions orS-functions. Finally, an output is constructed by combining the mappeddetail and regional contrast layers, i.e. the stretched first and secondsignal:

{tilde over (V)} ₁ ={tilde over (V)} _(D) +{tilde over (V)} _(RC),

{tilde over (V)}₁ is the combined stretched first and second signal. Inthis example this is done by combining the stretched first and secondsignal in combiner 5, in this example a simple adder.

This aspect of the invention improves the displayed image by reducingthe visible mismatch between regional contrast and detail contrast afterincrease of the dynamic range of the input signal.

A further problem occurring in HDR display is that the peak brightnessof new HDR displays is very high (e.g., the DR37-P by Brightside/Dolbyis reported to have a peak brightness of over 3000 cd/m2). Consequently,stretching the signal during display to the full dynamic range mayresult in unpleasantly bright scenes for some images. The range to whichthe input is stretched can be limited for instance to between 500 and1000 Nit, to avoid such unpleasant scenes but in this case the displays'capabilities are not fully exploited.

To address tins issue, in preferred embodiments of the invention afurther step is added to the algorithm. This preferred step isschematically shown in rectangle 6 in FIG. 1.

In order to take full advantage of an HDR displays' capabilities, smallspecular highlights are identified with which the remaining availabledynamic range, i.e. the range W₀ to W_(HDR) is populated (highlighting).Preferably bilateral grids are used, also as a form of low pass filter.Since bilateral grids involve constructing local histograms, thesehistograms can be used directly to identify regions with a small numberof bright pixels. The algorithm to perform identification of highlightsis in FIG. 1 schematically shown as function FHL(V_(in)) in identifier7. The data of the pixels that are identified as belonging to highlightsare enhanced by a factor in mapper 8 that brings them into the highestdynamic range, providing a signal V_(HL) highlighting small brightareas. The signals {tilde over (V)}₁ and V_(HL) are combined in combiner9 to provide an output signal V_(out). Mapping can be done by simplemultiplication or by more complex functions.

FIG. 2 schematically illustrates the various dynamic range enhancements.The input signal V_(in) has a dynamic range ranging from K_(LDR) toW_(LDR) This is mapped into a dynamic range ranging from K₀ to a W₀.Apart from that pixels that are identified as belonging to highlights,which include pixels in the highest input range, in FIG. 2 schematicallyindicated by arrow 7′, are identified by the hightlight identifier 7;for the pixels that are identified as belonging to highlights a mappingoperation in mapper 8 is performed wherein the data for the highlightsis mapped onto a larger dynamic range, covering in particular thehighest range of luminance HL, e.g. corresponding to the top part of thefull dynamic range of an HDR display device. This highest range ofluminance, reserved for highlights, is stretched to a maximum valueW_(HDR). This maximum value lies above the upper value W_(o) of thetarget dynamic range, which target range is kept to moderate luminancevalues to avoid unpleasant viewing conditions. “Ex” in FIG. 2illustrates schematically the extension of the dynamic range for {tildeover (V)}_(ν), “HL” schematically illustrates the higher dynamic rangereserved for the highlights signal V_(HL).

FIG. 3 illustrates a highlight identification algorithm.

The input signal is sent to the identifier 7. Those areas or blocks withpixels having a luminance I above a threshold value I_(threshold) and anumber n_(nv) of such high intensity pixels below a thresholdn_(threshold) are identified as highlights. Further examples are givenbelow.

As an example the following procedure can be followed: To includehighlighting in the processing flow, the intensity of the bilateral gridconstructed on the input signal is stretched both to the target dynamicrange [K₀-W₀], resulting in the grid B₀, and once to full dynamic rangeof the display [K_(HDR)-W_(HDR)], resulting in the grid B_(HDR). Thesetwo grids are adaptively mixed into the final grid B_(mapped) using amixing map M prior to slicing (interpolation):

B _(mapped) =MB _(HDR)+(1−M)B ₀.

Note that in this example ail the above operations are performed on agrid base, which is a heavily sub-sampled representation of the image,and hence are numerically inexpensive. The final output on fullresolution is constructed by means of slicing into the mapped bilateralgrid B_(mapped). To create the mixing map M, we adopt the followingapproach

-   1. Construct the regional cumulative histogram by summing the    existing local histograms,-   2. Establish the brightness I_(threshold) above which less than n    percent of image pixels reside. In other words the top n percent of    luminance values,-   3. Count (on a local basis) the amount of pixels n with intensities    higher than I_(threshold).-   4. Apply a morphological dilatation filter to create spatial    consistency between neighboring bins, resulting in a consistency    value C₀. If the consistency value is high, relatively large bright    areas are present, if the consistency value is small, small bright    areas are present,-   5. Compute a mixing factor M. The value of the mapping function M is    set to 1 for regions where the number of qualifying pixels is below    a predefined threshold T (small highlights and thus to be mixed in)    and falls off to 0 above this threshold to prevent large bright    image portions from becoming unpleasantly bright;

${M\left( C_{0} \right)} = \left\{ \begin{matrix}1 & {{{for}\mspace{14mu} C_{0}} \leq T} \\{{CLIP}\left( {{1 - \left( {{\left( {C_{0} - T} \right)/2}\; T} \right)},\left\lbrack {0,1} \right\rbrack} \right)} & {{{for}\mspace{14mu} C_{0}} > T}\end{matrix} \right.$

FIGS. 4 a to 4 f illustrate the effects of a dynamic range extensionalgorithm as described above. Shown are (a) a simulated LDR input image,as well as the (b) regional contrast layer and (c) detail layerextracted by means of bilateral grid filtering. The natural appearanceof the scene after extension is maintained by (d) extending only theregional contrast VRC to a user or manufacturer defined range [K₀-W₀].In (e), the intermediate output (sum of frames (c) and (d) is shown. In(f), the final mapped, output is shown in which small specularhighlights are identified to fill up the remaining available dynamicrange.

FIG. 5 illustrates the mixing map computed for the image of FIGS. 4 a to4 f. The scale on the right hand side gives the mixing factor M. Sometypical areas of mixing factor are indicated by arrows. The brightreflection in the water and the bright areas in the clouds are bothcorrectly detected as small highlights and are mapped on the fulldynamic range or near the full dynamic range of the HDR display device.Because the bright area in the sky is relatively large, a smaller weight(smaller mixing factor M) is attributed to this area to prevent it frombecoming unpleasant on a high-brightness HDR display. Second thelow-resolution nature of the map is clearly visible from its blockyappearance, because these operations are performed over localhistograms, not on full pixel resolution. In this preferred embodimentthe mixing map M is applied to the bilateral grids B₀ and B_(HDR). Thepixels themselves are constructed by slicing the final grid B_(mapped).As a result, only bright pixels in the enhanced area are affected by thehighlighting operation, but dark to mid-grey intensities remainunchanged, such that, the highlighting procedure is a selectiveoperation to fill up the top part HL of the available dynamic range. Thefinal output for the image of FIG. 3 a is shown in FIG. 3 f. The brightreflections in the water and in the clouds are now mapped to the peakbrightness of the display, (i.e. mixing factor M is near 1, these areasare indicated in FIG. 5 by the white arrows, while larger areas in thesky remain closer to the intermediate intensities of FIG. 3 e. Again, bypreventing over-enhancement of fine detail contrast a more naturalappearance of the scene is maintained.

FIGS. 6-8 show further examples of the performance of the proposeddynamic range extension method. FIGS. 6 a to 6 c form an illustration ofdynamic range extension. Shown are (a) the simulated LDR image and theextended output (b) without and (c) with highlighting. Again, thismethod is designed for giving a high performance on extremely bright HDRdisplays. Without such displays, we are here limited to simulations ofthe extension procedure. To this end, a LDR input is simulated and theextension procedure is used to restore the image to the full availablerange. Obviously, this simulation is imperfect and cannot provide arealistic appearance of the actual HDR display. Nevertheless FIGS. 6 ato 6 c illustrate the maintained balance between region and finecontrasts as well as the selective use of the peak brightness of thedisplay.

FIGS. 7 a to 7 d show further examples of dynamic range extension. Thesimulated LDR images 7 a and 7 c are shown on the left, the extended HDRversion including highlighting, images 7 b and 7 d, are shown on theright. In the lower example, ovals annotate highlighted areas. Thisexample illustrates that the large white areas in the snowy mountain arenot mapped to the peak brightness of the HDR display as this would beunpleasant. Instead, only small specular highlights are mapped to thefull brightness in a very selective procedure. In the upper example onlythe car headlights are mapped to the peak brightness.

FIGS. 8 a to 8 e provide further examples of dynamic range extension.The simulated LDR images, FIGS. 8 a and 8 c are shown on the left, theextended HDR versions including highlighting are shown on the right inFIGS. 8 a and 8 d.

In short the invention can be described as providing a method, unit anddisplay device in which the input image signal is split into a regionalcontrast signal and a detail signal, followed by stretching separatelythe dynamic ranges for both signals, wherein the dynamic range for theregional contrast signal is stretched with a higher stretch ratio thanthe dynamic range for the detail signal. Preferably the stretch ratiofor the detail signal is near 1 or preferably 1. In preferred embodimenthighlights are identified and for the highlights the dynamic range isstretched to an even higher degree than for the regional contrastsignal.

Stretching the regional contrast signal more than the detail signalreduces mismatch between enhancement of fine details relative toregional contrast and provides a more natural look. The more extremestretching of the dynamic range for highlighted areas maps thesehighlights in the top part of the dynamic range. This makes the imagesparkle without causing large overly bright areas, which would providefor unpleasant viewing.

The methods and system of the invention may be used in various mannersfor various purposes such as for instance to enables enhancementalgorithms and other video processing algorithms.

The invention is also embodied in a computer program comprising programcode means for performing a method according to the present invention,when executed on a computer.

The invention can be used in or for conversion units of image signalsand devices in which a conversion of image signals is used, such asdisplay devices, in particular in display devices with HDR capability.

In the claims, any reference signs placed between parentheses shall notbe construed as limiting the claim.

The word “comprising” does not exclude the presence of other elements orsteps than those listed in a claim. The invention may be implemented byany combination of features of various different preferred embodimentsas described above.

The invention is not limited to the above given examples, but can beexecuted in various ways.

For example:

The upper value W₀ may be made dependent on a number of parameters, themost important of which are

Color:

The maximum saturation, level for reflective red and blue colors isrelatively low compared to green and yellow. The value for W₀ is, inpreferred embodiments made dependent on the color, to avoid parts thatstart to glow rather than to blind.

Ambient illumination level:

In preferred embodiments the display device is provided with a lightsensor to sense the ambient illumination level. The output of theambient illumination sensor determines the upper value W₀, wherein thehigher the ambient illumination level, the higher the upper value W₀ isset FIG. 9 illustrates such an embodiment. The display device isprovided with a display screen 91. The output signal V_(out) determinesthe image displayed on the screen 91. The display device is furtherprovided with an ambient illumination sensor 92 for measuring theambient illumination. An output of this sensor is an input for thestretcher 3 for stretching the dynamic range of Y_(RC). The output ofthis sensor may also be coupled to identifier 7 and/or the mapper 8 fordetermining the highlights and/or stretching providing the dynamic rangefor the highlights. In this example the output of sensor 92 is feddirectly into identifier 7 and/or mapper 8. Within embodiments of theinvention the functional parameters of stretcher 3 and identifier 7and/or mapper 8 may be linked, so that the sensor signal could be sentto only one of the devices. Likewise, there could be a computer programcomprising a look-up table wherein functional parameters for stretcher3, such as the upper value of the dynamic range and/or distribution overthe dynamic range and/or for identifier 7 and/or mapper 8 as a functionof the sensor signal are stored. The output of the sensor is, in suchembodiment, an input for the computer program and the computer programcontrols the parameters for stretcher 3, identifier 7 and/or mapper 8.

graphics detection

In preferred embodiments a graphics detection unit is used to identitygraphics (such as logos, subtitles) to exclude them from enhancementand/or highlighting.

The invention is also embodied in various systems:

The image conversion unit can also form part of an image processingapparatus of various kinds.

For instance, the conversion unit for performing the conversion can bepart of a display device, as in FIG. 9.

“Conversion unit” is to be broadly interpreted as any means, includingsoftware, hardware or any combination thereof for performing the methodof conversion.

The conversion unit can also be part of for instance a recording device.One can record an image or video, wherein the recording device isprovided with information on the capabilities of the display devices.The recording devices applies, in real time or of line, the methodaccording to the invention, matching the dynamic range W₀-K₀ and/orW_(HDR)-K_(HDR) to the capabilities of the display screen. The improvedimage or video can then be displayed, either in real time, orafterwards.

In a variation to this system, the software may be on some server on theinternet. The user sends the image data of images or videos he/she hasto an internet site and provides the internet site with details on thedynamic range capabilities of the display device he/she has. Thisdynamic range information can be explicit, for instance by specifyingthe dynamic range, or implicit, for instance by specifying the displaydevice he/she has, or even without the user noticing it, since the typeof display is automatically checked. At the server it is checkedwhether, given the capabilities of the display device, applying themethod of the invention to the input image data an improved image orvideo is produced. If the answer is positive the method of the inventionis applied to the input image data, and, after having received paymentfor the service, the improved output image data, matched to thecapabilities of the HDR display, is sent back to the user.

This embodiment allows a user to upgrade his/her “old” image or videos,to make full use of the HDR capabilities of his/her newly bought HDRdisplay without forcing the user to buy a specific conversion unit.

In “pay per view” systems, for instance to watch sport, the user may begiven the option of buying standard quality, or upgraded quality,wherein the upgraded quality is matched to the dynamic range of thespecific HDR display device he/she has.

1. A method for converting input image data of an image into output image data comprising the acts of: splitting the input image data into a first signal having a first dynamic range and a second signal having a second dynamic range; stretching the first dynamic range of the first signal to provide a stretched first signal having a stretched dynamic range, wherein the stretched dynamic range of the stretched first signal is higher than the second dynamic range of the second signal; combining the stretched first signal and the second signal to from a combined signal having a dynamic range bound by an upper value; analyzing the input image data to identity groups of pixels forming highlights in the image; converting pixel data for said identified groups of pixels into a third signal having a dynamic range extending upwards above the upper value to an upper maximum pixel value; combining the third signal with the combined signal to form the output image data; and outputting the output image data.
 2. The method of claim 1, wherein the second signal is made by subtracting the first signal from the image input data.
 3. The method of claim 1, further comprising the act of stretching the second signal to form a stretched second signal for combining with the stretched first to form a stretched combined signal.
 4. The method of claim 1, wherein an upper value of the dynamic range for the combined signal lies in a range corresponding to light intensities when displayed on a display of 500 to 1000 Nit.
 5. The method of claim 1, wherein the upper maximum pixel value lies in a range corresponding to light intensities when displayed on a display of above 1000 Nit.
 6. A non-transitory computer readable medium comprising computer instructions which, when executed by a computer, configure the computer to-perform the acts of: splitting input image data of an image into a first signal having a first dynamic range and a second signal having a second dynamic range; stretching the first dynamic- range of the first signal to provide a stretched first signal having a stretched dynamic range, wherein the stretched dynamic range of the stretched first signal is higher than the second dynamic range of the second signal; combining the stretched first signal and the second signal to from a combined signal having a dynamic range bound by an upper value; analyzing the input image data to identify groups of pixels forming highlights in the image; converting pixel data for said identified groups of pixels into a third signal having a dynamic range extending upwards above the upper value to an upper maximum pixel value; combining the third signal with the combined signal to form output image data; and outputting the output image data.
 7. The non-transitory computer readable medium of claim 6, wherein the second signal is made by subtracting the first signal from the image input data.
 8. An image conversion unit for converting input image data of an image into output image data, comprising: a splitter configured to split the input image data into a first signal having a first dynamic range and a second signal having a second dynamic range; a stretcher configured to stretch the first dynamic range of the first signal to provide a stretched first signal having a stretched dynamic range; wherein the stretched dynamic range of the stretched first signal is higher than the second dynamic range of the second signal; a combiner configured to combine the stretched signal and the second signal to from a combined signal having a dynamic range hound by an upper value; an identifier configured to analyze the input image signal and to identify groups of pixels forming highlights in the image; a mapper configured to map pixel data for said identified groups of pixels into a third signal such that the third signal covers a dynamic range extending upwards above the said upper value to an upper maximum pixel value; and a further combiner configured to combine the third signal with the combined signal to form the output image data.
 9. The image conversion unit of claim 8, wherein the second signal is made by subtracting the first signal from the image input data.
 10. The image conversion unit of claim 8, wherein the stretcher is arranged such that the upper value of the dynamic range for the combined signal lies in a range corresponding to light intensifies when displayed on a display of 500 to 1000 Nit.
 11. The image conversion unit of claim 8, wherein the mapper is arranged such that the upper maximum pixel value lies in a range corresponding to light intensities when displayed on a display of above 1000 Nit.
 12. A display device comprising an image processing apparatus comprising; a receiver configured to receive input image data of an image; an image conversion unit configured to convert the input image data into output image data; and a display screen configured to render the output image data, wherein, the image conversion unit comprises: a splitter configured to split the input image data into a first signal having a first dynamic range and a second signal having a second dynamic range; a stretcher configured to stretch the first dynamic range of the first signal to provide a stretched first signal having a stretched dynamic range, wherein the stretched dynamic range of the stretched first signal is higher than the second dynamic range of the second signal; a combiner configured to combine the stretched signal and the second signal to from a combined signal having a dynamic range bound by an upper value; an identifier configured to analyze the input image data and to identify groups of pixels forming highlights in the image; a mapper configured to map pixel data for said identified groups of pixels into a third signal such that the third signal covers a dynamic range extending upwards above the said upper value to an upper maximum pixel value; and a further combiner configured to combine the third signal with the combined signal to form the output image data.
 13. The display device of claim 12, wherein the upper maximum pixel value corresponds to a value at or near a maximum of the dynamic range of the display screen.
 14. The display device of claim 12, wherein the display device comprises an ambient illumination sensor providing an output, wherein the output of the ambient illumination sensor is an input for the stretcher.
 15. An image conversion unit for converting input image data of an image into output image data, comprising: a splitter configured to split the input image data into a first signal and a second signal; a stretcher configured to stretch the first signal and the second signal differently, wherein a dynamic range of the first signal is stretched to a first stretched dynamic range and a dynamic range of the second signal is stretched to a second stretched dynamic range, the first stretched dynamic range of the stretched first signal being higher than the second dynamic stretched range of the second signal; and a combiner configured to combine the stretched first and second signals to from a combined signal having a dynamic range bound by art upper value, wherein the combined signal is combined with a third signal having a third dynamic range higher than the upper value for forming the output image data.
 16. The image conversion unit of claim 15, further comprising: an identifier configured to analyze the input image signal and to identify groups of pixels forming highlights in the image; a mapper configured to map pixel data for said identified groups of pixels into the third signal; and a further combiner configured to combine the third signal with the combined signal to form the output image data.
 17. The image eon version unit of claim 15, wherein the second signal is formed by subtracting the first signal from the image input data.
 18. An image conversion unit for converting input image data of an image into output image data, comprising; a splitter configured to split the input image data into a first signal having a first dynamic range and a second signal having a second dynamic range; a stretcher configured to stretch the first signal for changing the first dynamic range to a stretched dynamic range which is higher than the second dynamic range of the second signal; and a combiner configured to combine the stretched first and second signals to from a combined signal having a dynamic range bound by an upper value, wherein the combined signal is combined with a third signal having a third dynamic range higher than the upper value for forming the output image data.
 19. The image conversion unit of claim 18, further comprising; an identifier configured to analyze the input image signal and to identify groups of pixels forming highlights in the image; a mapper configured to map pixel data for said identified groups of pixels into the third signal; and a further combiner configured to combine the third signal with the combined signal to form the output image data.
 20. The image conversion unit of claim 18, wherein the second signal is formed by subtracting the first signal from the image input data. 